Method of producing voice data method of playing back voice data, method of playing back speeded-up voice data, storage medium, method of assisting memorization, method of assisting learning a language, and computer program

ABSTRACT

A computer-assisted method of assisting a user of a computer to memorize includes steps of displaying an image indicating a content to be memorized, on a left-hand side of a computer screen for a particular period; displaying text data indicating language information related to the image displayed in this displaying step, on a right-hand side of the computer screen for a particular period; and playing back a voice pronouncing the text data displayed in the text data displaying step. The image displaying step, the text displaying step, and the voice playing back step are performed repeatedly, thereby assisting the user of the computer to memorize.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method of, by using a computer,producing speeded-up voice data or slowed-down data, a method of playingback such voice data, a method of assisting a user of a computer tomemorize or learn a language, and a computer program for executing sucha method.

2. Description of the Related Art

It is known in the art to play back voice data at a high speed (2, 3, or4 times higher than the normal speed) using a dedicated tape recorder. Aflash card is also known as a tool of learning a language. For example,a teacher of English shows an English word written on a thick. papercard with a size of about 10 cm in height and 25 cm in width to pupilsfor a very short time, and pupils pronounces the English word shown. Itis also known to employ a card game in teaching English to littlechildren. In this teaching technique, pictures and corresponding Englishwords are drawn on respective cards, and children look for a particularcard.

However, in the conventional quick playback system, voice data recordedon a tape is processed, and thus this technique is not suitable for usewith a multimedia system such as that using the Internet.

The inventor of the present invention has read a scientific newspaperarticle entitled “Eyes cheat Ears” (Nihon Keizai Shimbun, Sunday, Jun.23, 2002, p. 28). In this article, various examples of cheating betweenan image and a sound are described. For example, if a sound is generatedtwice each time light is emitted once, human eyes misperceive that lightis emitted twice. In another example, after seeing a rectangular imagemoving away repeatedly, a sound with a constant volume is misperceivedas increasing in volume. In a moving picture in which two balls movetoward each other and pass each other, if a collision sound isgenerated, human eyes misperceive that the two balls are repelled fromeach other. From this article, the inventor of the present invention hasgotten an excellent idea that effects of quick playback can be enhancedby displaying an image when a voice is being played back at an increasedspeed.

The inventor also has got idea of realizing flash cards on a computerthereby realizing an improved method of assisting memorization.Furthermore, the inventor has got an idea of improving a manner ofdisplaying an image in a computer-assisted memorization system, byapplying cerebric physiological effects, such as those described forexample in “Brain System” (edited by Makoto Iwata and published byNatsume-sha Publishing Co.).

The description on pages 74 to page 81 of “Brain System” published byNatsume-sha can be summarized as follows. A right-hand portion of abrain is associated with left-hand portions of a human body, while alight-hand portion of the brain is associated with right-hand portionsof the human body. Information from a left eye is treated by theright-hand portion of the brain, while information from a right eye istreated by the left-hand portion of the brain. Processing associatedwith languages is performed by the left-hand portion of the brain, whileprocessing associated with images is performed by the right-hand portionof the brain. Many examples of experiments of manners in which brainswork are also described. Of those, a description of effects of showingan image to an examinee for a very short time such as 0.1 sec or shorteris instructive.

The inventor of the present invention has got an idea of realizing amethod of assisting a user to memorize a great amount of information ina short time, on the basis of the quick playback technique and thecerebric physiological effects.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a quick playback methodincluding a digital process using a computer. Another object of thepresent invention is to provide a method of enhancing effects of quickplayback by using an image. Another object of the present invention isto provide a computer-assisted method. of assisting a user to memorize alarge amount of information.

According to an aspect of the present invention, to achieve the aboveobjects, there is provided a method of producing speeded-up voice databy digitally processing original voice data on a computer, the methodcomprising: a block sampling step of producing a plurality of blockswith an equal length by sampling the original voice data at apredetermined sampling frequency; a fading-in/out step of fading in orout end portions of the respective blocks produced in the block samplingstep; and a block-edge superimposing step of superimposing end portionsof respective adjacent blocks subjected to fading-in/out in thefading-in/out step.

In the fading-in/out step, the fading-in/out of end portions of blocksmay be performed by multiplying the end portions of the blocks by afunction.

Preferably, in the fading-in/out step, each end portion of blocks isfaded in or out over a length equal to the length of each end portion tobe superimposed in the block-edge superimposing step.

The length of each of end portions of respective adjacent blockssuperimposed in the block-edge superimposing step may be set to be equalto (n−1)/n times the block length thereby producing voice data speededup by a factor n times the normal speed (wherein n is a rational numberequal to or greater than 1).

According to another aspect of the present invention, there is provideda method of producing voice data for use in slow playback, the methodcomprising a block sampling step of producing a plurality of blocks withan equal length by sampling the original voice data at a predeterminedsampling frequency; and a block insertion step of inserting a new blockbetween adjacent blocks produced in the block sampling step.

The method may further comprise the steps of fading in or out each endportion of blocks to be inserted in the block insertion step; andsuperimposing faded-in/out end portions of respective blocks withadjacent blocks thereby producing slowed-down voice data.

The fading-in/out of end portions of blocks may be performed bymultiplying the end portions of the blocks by a function.

Preferably, the length of each block inserted in the block insertionstep is equal to the length of each faded-in/out end portion of adjacentblocks.

The length of each block inserted in the block insertion step may be setto be equal to m times the length of each block sampled in the blocksampling step thereby producing voice data for use in slow playback at aspeed equal to 1/(1+m) times the normal speed (wherein m is a rationalnumber equal to or smaller than 1).

In playing back the produced speeded-up voice data, it is preferable tofirst playing back voice data speeded up by a smaller factor, and thenplaying back voice data speeded up by a greater factor, and it is alsopreferable to first playing back voice data speeded up by a greaterfactor, and then playing back voice data speeded up by a smaller factor.

Furthermore, when the speeded-up voice data is played back, it is alsopreferable to play back a natural sound at a normal speed such that thenatural sound is superimposed on the speeded-up voice data.

When slowed-down voice data is played back, it is preferable to firstplay back voice data slowed down by a greater slowdown factor and thenplay back voice data slowed down by a smaller slowdown factor.Alternatively, it is also preferable to first play back voice dataslowed down by a smaller slowdown factor and then play back voice dataslowed down by a greater slowdown factor.

Furthermore, when the slowed-down voice data is played back, it is alsopreferable to play back a natural sound at a normal speed such that thenatural sound is superimposed on the slowed-down voice data

A user of a computer may install, onto his/her computer from a storagemedium, a computer program for executing a method of producingspeeded-up or slowed-down voice data thereby making it possible for theuser to produce speeded-up or slowed-down voice data from voice datahe/she owns.

For use by end users who do not use a computer, a storage medium onwhich speeded-up voice data has been stored using a proper program maybe provided.

When speeded-up voice data is played back, it is further preferable todisplay an image or text linked to the speeded-up voice data.

According to another aspect of the present invention, there is provideda method of playing back speeded-up voice data, comprising the steps of:playing back the speeded-up voice data while playing back a naturalsound at a normal speed in a superimposed fashion; and displaying amoving image corresponding to the speeded-up voice data insynchronization with the speeded-up voice data.

According to still another aspect of the present invention, there isprovided a method of playing back speeded-up voice data, comprising thesteps of equalizing the speeded-up voice data such that resultantequalized voice data has frequency components in a range which would behearable by an unborn baby in a womb; and playing back the equalizedspeeded-up voice data.

According to still another aspect of the present invention, there isprovided a method of playing back speeded-up voice data, comprising thesteps of equalizing the speeded-up voice data so as to emphasizefrequency components in a range used by a foreign language; and playingback the equalized speeded-up voice data.

According to still another aspect of the present invention, there isprovided a method of playing back speeded-up voice data, comprising thesteps of playing back the speeded-up voice data while displaying animage moving in a scene of a natural environment.

In this method of playing back speeded-up voice data, the speed at whichthe image moves in the scene of the natural environment may be increaseddepending on the speed of the speeded-up voice data.

According to still another aspect of the present invention, there isprovided a computer-assisted method of assisting a user to memorize,comprising the steps of: displaying an image indicating a content to bememorized, on a left-hand side of a computer screen for a predeterminedperiod of time; and displaying text data indicating language informationrelated to the image displayed in the image display step, on aright-hand side of the computer screen for a predetermined period oftime, wherein the above two steps are performed repeatedly tosuccessively display a plurality of images and text data therebyassisting the user of the computer to memorize.

According to still another aspect of the present invention, there isprovided a computer-assisted method of assisting a user to memorize,comprising the steps of: displaying an image indicating a content to bememorized, on a left-hand side of a computer screen for a predeterminedperiod of time; displaying text data indicating language informationrelated to the image displayed in the image display step, on aright-hand side of the computer screen for a predetermined period oftime, and playing back a voice pronouncing the text data displayed inthe text data displaying step, wherein the above three steps areperformed repeatedly to successively display a plurality of images andtext data and play back voices thereby assisting the user of thecomputer to memorize.

The method may further comprise a time parameter adjustment step ofadjusting, by the user of the computer, a cycle time of repeatedlydisplaying images in the image display step, the length of an imagedisplay period during which each image is displayed in the image displaystep, the timing of starting displaying text data in the text displaystep, and the length of a text display period during which text data isdisplayed in the text data displaying step, wherein once the adjustmentin the time adjustment step is performed by the user of the computer,displaying of the image in the image display step and displaying of thetext data in the text data display step are performed in accordance withthe conditions set in the time adjustment step.

Alternatively, the method may further comprise a time parameteradjustment step of adjusting, by the user of the computer, a cycle timeof repeatedly displaying images in the image display step, the length ofan image display period during which each image is displayed in theimage display step, the timing of starting displaying text data in thetext display step, the length of a text display period during which textdata is displayed in the text data displaying step, and the timing ofstarting playing back the voice in the voice playback step, wherein oncethe adjustment in the time adjustment step is performed by the user ofthe computer, displaying of the image in the image display step,displaying of the text data in the text data display step, and playbackof the voice in the voice playback step are performed in accordance withthe conditions set in the time adjustment step.

The method may further comprise a voice playback speed adjustment stepof adjusting, by the user of the computer, the speed of playing back thevoice in the voice playback step, wherein once the adjustment in thevoice playback speed adjustment step is performed by the user of thecomputer, playback of the voice in the voice playback step. is performedin accordance with the condition set in the adjustment step.

The method may further comprise an area overlap ratio setting step ofsetting, by the user of the computer, an. area overlap ratio indicatingthe degree of overlapping between an area in which images are displayedin the image display step and an area in which text data is displayed inthe text display step, wherein once the setting in the area overlapration setting step is performed by the user of the computer, displayingof the image in the image display step and displaying of the text datain the text data display step are performed in accordance with thecondition set in the setting step.

The method may further comprise playing back a natural sound or musicsuch that the natural sound or music is superimposed on the voice playedback at the adjusted speed in the voice playback step.

In this method, the plurality of image data may be stored in the form ofa database, and the method may further comprise the step of extractingimage data categorized by synonym, antonym, grade, type, or the like,wherein the extracted image data may be displayed in the steps performedrepeatedly.

When the method of assisting a user to memorize is being executed, anatural sound or music may be played back. The method of assisting auser to memorize may be executed on a computer.

In this method of assisting a user to memorize, the computer may beincluded in a personal digital assistance (PDA) device, a game machine,or a similar device. The method of assisting a user to memorize may beemployed to assist a user to learn a language.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A to 1D are conceptual diagrams illustrating a manner ofproducing speeded-up voice data;

FIGS. 2A and 2B are conceptual diagrams showing a fading-in/out process;

FIGS. 3A to 3C are diagrams conceptually showing a process of producingvoice data speeded up by a factor of 2;

FIGS. 4A to 4C are diagrams conceptually showing a process of producingvoice data speeded up by a factor of 3;

FIGS. 5A to 5E are diagrams conceptually showing a process of producingvoice data speeded up by a factor of 0.5 (that is, slowed down by afactor of 2);

FIG. 6 is a flow chart showing a process of speeded-up voice data;

FIG. 7 is a flow chart showing a process of playing back speeded-upvoice data stored on a storage medium;

FIG. 8 is a flow chart showing a process of setting conditions underwhich to play back speeded-up voice data to perform practice byexecuting a computer program;

FIG. 9 is a hard copy of a screen for setting conditions under which toplay back speeded-up voice data to perform practice by executing acomputer program;

FIG. 10 is a flow chart showing a process of displaying a moving imageof, for example, a bird;

FIG. 11 is a flow chart showing a process of setting a mode;

FIG. 12 is a flow chart showing a process including an equalizationstep;

FIG. 13 is a flow chart showing a process of setting an equalizationmode;

FIG. 14 is a diagram showing passbands of various languages;

FIG. 15 is a diagram showing the relationship between parts of a humanbody and frequencies of sounds;

FIG. 16 is a diagram showing an initial screen displayed at thebeginning of a program;

FIG. 17 is a hard copy of a setting screen which is displayed inresponse to clicking a setting button;

FIG. 18 is a hard copy of a screen on which an image of a “double-deckerbus” is displayed on the left-hand side, wherein the left-right imageoverlapping ratio is set to 30%;

FIG. 19 is a hard copy of a screen on which text data of “double-deckerbus” is displayed on the right-hand side, wherein the left-right imageoverlapping ratio is set to 30%, and Japanese is specified as thelanguage;

FIG. 20 is a hard copy of a screen on which an image of a “horse-drawncarriage” is displayed on the left-hand side, wherein the left-rightimage overlapping ratio is set to 0%;

FIG. 21 is a hard copy of a screen on which text data of “horse-drawncarriage” is displayed on the right-hand side, wherein the left-rightimage overlapping ratio is set to 0%, and Japanese is specified as thelanguage;

FIG. 22 is a hard copy of a screen on which an image of a “jeep” isdisplayed on the left-hand side, wherein the left-right imageoverlapping ratio is set to 100%;

FIG. 23 is a hard copy of a screen on which text data of “jeep” isdisplayed on the right-hand side, wherein the left-right imageoverlapping ratio is set to 100%, and Japanese is specified as thelanguage;

FIG. 24 is a hard copy of a screen on which an image of a “scooter” isdisplayed on the left-hand side, wherein the left-right imageoverlapping ratio is set to 71%;

FIG. 25 is a hard copy of a screen on which text data of “scooter” isdisplayed on the right-hand side, wherein the left-right imageoverlapping ratio is set to 71%, and Japanese is specified as thelanguage;

FIG. 26 is a hard copy of a screen on which an image of a “van” isdisplayed on the left-hand side, wherein the left-right imageoverlapping ratio is set to 16%;

FIG. 27 is a hard copy of a screen on which text data of “van” isdisplayed on the right-hand side, wherein the left-right imageoverlapping ratio is set to 16%, and Japanese is specified as thelanguage;

FIG. 28 is a hard copy of a screen on which an image of a “cable car” isdisplayed on the left-hand side, wherein the left-right imageoverlapping ratio is set to 82%;

FIG. 29 is a hard copy of a screen on which text data of “cable car” isdisplayed on the right-hand side, wherein the left-right imageoverlapping ratio is set to 82%, and Japanese is specified as thelanguage;

FIG. 30 is a flow chart of a program according to the present invention;

FIG. 31 is a flow chart showing a process performed by a user to setvarious conditions;

FIG. 32 is a time chart associated with an operation in which voice dataspeeded up by a factor of 2 has a length shorter than a specified cardturning interval;

FIG. 33 is a time chart associated with an operation in which voice dataspeeded up by a factor of 2 has a length longer than a specified cardturning interval, and the card turning interval is automaticallyincreased by the program;

FIG. 34 is a conceptual diagram showing a method of assisting a user tomemorize or learn a language according to the present invention;

FIG. 35 is a diagram conceptually showing image data, text data, andvoice data, which are linked to each other and used in the method ofassisting a user to memorize or learn a language according to thepresent invention;

FIG. 36 is a diagram showing a screen for selecting a CD storing music(or a natural sound) to be played back in execution of the method ofassisting a user to memorize or learn a language according to thepresent invention, wherein in this specific example, a message “NO MUSICCD” is displayed to indicate, that a selectable music CD is not found ora content of a CD is not of music although the CD is found;

FIG. 37 is a diagram showing a screen for selecting a CD storing music(or a natural sound) to be played back in execution of the method ofassisting a user to memorize or learn a language according to thepresent invention, wherein in this specific example, a music CD is foundand the number of pieces stored on the music CD is displayed (20 piecesare stored on the music CD, in the example shown);

FIG. 38 is a diagram showing a screen for selecting a CD storing music(or a natural sound) to be played back in execution of the method ofassisting a user to memorize or learn a language according to thepresent invention, wherein in this specific example, an invertedtriangular button on the right-hand end of a field in which a message“20 pieces of music is stored on a music CD” is indicated is clicked toopen a pull-down list of selectable 20 pieces of music;

FIG. 39 is a diagram showing a screen for selecting a CD storing music(or a natural sound) to be played back in execution of the method ofassisting a user to memorize or learn a language according to thepresent invention, wherein in this specific example, music with a lengthof 2 min 47 sec stored on track 3 is selected, and a “repeat singlepiece” button becomes clickable;

FIG. 40 is a diagram showing a screen for selecting a CD storing music(or a natural sound) to be played back in execution of the method ofassisting a user to memorize or learn a language according to thepresent invention, wherein in this specific example, all 20 pieces ofmusic are selected and a “shuffle” button is clicked to specify that theselected 20 pieces of music should be played back at random;

FIG. 41 is a diagram showing files used in the program of the method ofassisting a user to memorize or learn a language, wherein in thisspecific example, image data and voice data are stored in a folder“ClipArts” which has sub-folders “animals” and “vehicles” each of whichhas a sub-folder “voice” for storing voice data. Each folder “voice” hasa sub-folder “2x” for storing voice data speeded up by a factor of 2.

FIG. 42 is a diagram showing a screen displayed when the program of themethod of assisting a user to memorize or learn a language is started,wherein in this specific example, the image-text overlapping ratio isset to 100%;

FIG. 43 is a hard copy of a screen for performing settings associatedwith a drill;

FIG. 44 is a hard copy of a window in which a list of drills isdisplayed;

FIG. 45 is a diagram showing a drill setting screen, wherein in thisspecific example, image data is called on the screen and text data andvoice data associated are linked to the image data;

FIG. 46 is a diagram showing a screen for installing image data;

FIG. 47 is a hard copy of a screen for setting conditions associatedwith a display cycle time, an image display period, a text display.time, and a text delete timing;

FIG. 48 is a diagram showing an initial screen for installing theprogram of the method of assisting memorization according to the presentinvention onto a computer;

FIG. 49 is a diagram showing a screen for specifying a directory intowhich to install the program;

FIG. 50 is a diagram showing a screen for specifying a program groupname into which to install the program;

FIG. 51 is a diagram showing a window in which a message indicating thecompletion of the installation is displayed;

FIG. 52 is a diagram showing a manner in which the program is started;

FIG. 53 is a diagram showing a manner in which the program isuninstalled;

FIG. 54 is a diagram showing a screen in which condition 1 is set;

FIG. 55 is a diagram showing a screen in which condition 2 is set;

FIG. 56 is a diagram showing a screen in which condition 3 is set;

FIG. 57 is a diagram showing a screen in which condition 4 is set;

FIG. 58 is a diagram showing a screen in which condition 5 is set;

FIG. 59 is a diagram showing a screen in which condition 6 is set;

FIG. 60 is a diagram showing a screen in which condition 7 is set;

FIG. 61 is a diagram showing a screen in which condition 8 is set;

FIG. 62 is a diagram showing a screen in which condition 9 is set; and

FIG. 63 is a block diagram showing an example of a hardwareconfiguration of a computer used to realize the method of assistingmemorization according to the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention is described in further detail below withreference to preferred embodiments in conjunction with the accompanyingdrawing. FIG. 63 is a block diagram showing an example of a computer forexecuting a method of producing voice data, a method of playing backvoice data, a method of playing back speeded-up voice data, or a methodof assisting a user to memorize or learn a language, according to thepresent invention. As shown in FIG. 63, the computer includes a centralprocessing unit (CPU) 10. The computer further includes a read onlymemory (ROM) 11, a random access memory (RAM) 12, a video RAM 13, a harddisk drive (HDD) 20, and a CD-ROM drive (CDD) 21, wherein thosecomponents are connected to the CPU 10 via a bus. The computer alsoincludes a display 23 connected to the video RAM 13. As for basicsoftware for this computer, for example, basic software for personalcomputer available from Microsoft Corp. in the USA may be employed. TheCD-ROM drive 21 is used to install a program onto the computer and isalso used to read background music data, to be used in the presentinvention, from-a music CD.

FIGS. 1A to 1D are conceptual diagrams illustrating a manner ofproducing speeded-up voice data. FIG. 1A shows blocks produced bysampling original voice data at, for example, 11 kHz and combining a fewhundred (for example, 400 to 1000) sampled data into each block. Allblocks are equal in size. FIG. 1B shows a manner in which adjacentblocks are partially overlapped with each over by a particular amount.To produce voice data speeded up by a factor of 2, one-half of eachblock is overlapped with adjacent blocks. On the other hand, to producevoice data speeded up by a factor of 3, two thirds of each block isoverlapped with adjacent blocks. In general, if (n−1)/n of each block isoverlapped with adjacent blocks, voice data speeded up by a factor of nis obtained. FIG. 1C shows a manner in which each block is faded in orout in its end portions to be overlapped. By performing fading in/out,each block is converted such that the amplitude becomes zero at bothends and the amplitude gradually increases toward the center of theblock. The fading in/out may be performed by multiplying a linearfunction or another known function such as a trigonometric function or alogarithmic function. FIG. 1D shows a manner in which the faded-in/outblocks shown in FIG. 1C are connected to each other in an overlappedfashion thereby producing speeded-up voice data. The overlapping causesthe total length of data to decrease, and thus the resultant data can beplayed back at a corresponding high speed.

FIGS. 2A and 2B show a manner in which the fading in/out is performed.That is, FIG. 2A shows a manner in which a trailing end portion of ablock is faded out, and FIG. 2B shows a manner in which a leading endportion of a block is faded in. In FIGS. 2A and 2B, the fading in/out isperformed by multiplying an original amplitude denoted by arrows in thefigures by a particular function. In the example shown in FIGS. 2A and2B, a linear function, which increases from zero at an edge of eachblock to 1 at an opposite end of the overlapping interval, is employedas the particular function. In a case in which the fading in/out isperformed using a trigonometric function, (1+cos x)/2 may be employedfor fading out at the end of each block, and (1−cos x)/2 may be employedfor fading in at the start of each block (wherein, for simplicity, bothend portions, to be overlapped, of each block are assumed to have alength equal to π). Herein, (1−cos x)/2 and (1+cos x)/2 are functionswhich monotonically increase from 0 to 1 or decrease from 1 to 0 in aninterval of x from 0 to n.

FIGS. 3A to 3C are diagrams conceptually showing a manner in which voicedata speeded up by a factor of 2 is produced. To produce voice dataspeeded up by a factor of 2, as described earlier, one-half of eachblock is overlapped with adjacent blocks. FIG. 3A shows a manner inwhich original data is sampled and grouped into blocks with an equallength. FIG. 3B shows a manner in which one-half of each block isoverlapped with adjacent blocks. FIG. 3C shows resultant speeded-upvoice data produced in the above-described manner.

FIGS. 4A to 4C are diagrams conceptually showing a manner in which voicedata speeded up by a factor of 3 is produced. To produce 3× speeded-upvoice data, two thirds of each block is overlapped with adjacent blocks.FIG. 4A shows a manner in which original data is sampled and groupedinto blocks with an equal length. FIG. 4B shows a manner in which twothirds of each block is overlapped with adjacent blocks. FIG. 4C showsresultant speeded-up voice data produced in the above-described manner.The manner of producing voice data speeded up by a factor of 2 or 3 hasbeen described above. Similarly, voice data speeded up by a greaterfactor such as 4 or 5 can also be produced. Such voice data speeded upby a factor of 4 or 5 can have quality high enough for practical use.Furthermore, it is also possible to produce voice data speeded-up by anarbitrary factor between the above-described speed-up factors, as longas the factor is equal to a rational number. For example, the speed-upfactor of voice data may be set to 2.7, 3.8, or any more preciselyspecified value. Also in the case in which such a value is specified asthe speed-up factor of voice data, the block overlap ratio may be givenby (n−1)/n.

FIGS. 5A to 5E are diagrams conceptually showing a manner in which voicedata speeded up by a factor of 0.5 (that is, slowed down by a factor of2) is produced. To produce slowed-down voice data, original data isfirst sampled and grouped into blocks as shown in FIG. 5A. Thereafter,spaces with a particular length are inserted between respective adjacentblocks as shown in FIG. 5B. Furthermore, as shown in FIG. 5C, a partwith a particular length of each immediately preceding block is copiedand fade-out processing is performed on the copied part, and theresultant part is put in a corresponding space. Similarly, as shown inFIG. 5D, a part with a particular length of each immediately followingblock is copied and fade-in processing is performed on the copied part,and the resultant part is put in a corresponding space. If blocks shownin FIGS. 5B, 5C and 5D are superimposed on each other, slowed-down voicedata is produced as shown in FIG. 5E. Herein, in FIG. 5B, if the lengthof each space created between adjacent original blocks is set to be mtimes the length of each original block (where m is a rational numberequal to or smaller than 1), the speed factor of the resultant voicedata becomes equal to 1/(1+m). For example, if m is set to 1, the speedfactor becomes equal to 0.5. If m is set to 0.5, the speed factorbecomes 0.67. In a case in which m is set to 0.25, the speed factorbecomes 0.8.

FIG. 6 is a flow chart showing a process of producing speeded-up voicedata. In step 610, original voice data is sampled. In step 620, sampleddata are grouped into blocks. In step 630, end portions of each blockare faded in or out, depending on a selected speed-up factor. In step640, blocks are connected to each other such that end portions ofadjacent blocks are overlapped with each other thereby producingspeeded-up voice data.

FIG. 7 is a flow chart showing a process of playing back speeded-upvoice data stored on a storage medium. In a case in which speeded-upvoice data already stored on storage medium according to the presentinvention is played back by an end user, the storage medium is first seton an audio player (such as a music CD player) (step 710). A startbutton is then pressed (step 720). If it is desired to play back thespeeded-up voice data repeatedly, a repeat button is pressed (step 730).

FIG. 8 is a flow chart showing a process of setting conditions underwhich to play back speeded-up voice data to perform practice executing acomputer program. In this process, the setting is performed by an enduser depending on various factors such as a grade or a degree ofprogress in practice. The process includes selecting a quick playbackmode (step 810), selecting a natural sound (step 820), setting apractice time (step 830), and starting (step 840). The respective stepsare described in detail below with reference to FIG. 9.

FIG. 9 is a hard copy of a computer screen used to set conditions underwhich to play back speeded-up voice data to perform practice byexecuting a computer program. Three sets of conditions frequently usedby the user are registered as quick playback modes A, B, and C,respectively. For example, playback mode A may be defined such thatplayback is started from a 3× speed and the speed is successivelydecreased to 2.5×, 2×, and 1.5×, and playback mode B may be defined suchthat playback is started from a 1× speed and the speed is successivelyincreased 1.5×, 2×, 2.5×, and 3× and then decreased to a 2× speed. Adesired playback mode can be specified by selecting a correspondingradio button on the screen. After voice data is selected by inputting afile name of an voice data file (or selecting by means of browsing) andother conditions associated with a practice time, a speed-up factor,etc., if a start button is clicked, the program executes the process ofproducing speeded-up voice data and playing back it in accordance withthe specified conditions.

The natural sounds selectable by a user may include a wind sound, a wavesound, a bird's singing, and a sound in a wood. It is preferable to playback a natural sound at a normal speed regardless of whether a voice isspeeded up or slowed down. The inventor of the present invention hasexperimentally found that superimposing of a natural sound at the normalspeed on a voice played back at an increased speed makes it easier for auser to listen to the speeded-up voice. From this point of view, it isdesirable that, after speeded-up voice data is produced, a natural soundat a normal speed is superimposed on the speeded-up voice data. Naturalsounds include frequency components which are not included in humanvoices. Thus, it is desirable that an audio system including anamplifier and a speaker used to reproduce sounds be capable ofreproducing frequency components over a wide range up to 25 Hz orhigher.

Voice data speeded-up by various factors such as 1, 2, 3, and 4 may beproduced in accordance with the present invention and resultant voicedata may be stored on a storage medium such as a CD-ROM in such a mannerthat speeded-up voice data are stored in the same order as the order inwhich voice data are played back, and the resultant storage medium maybe supplied to an end user. Although many beginners can easily listen tovoice data speeded up by a factor up to about 2.7, voice data speeded upby a factor higher than 2.7 is too fast for most beginners. Thus, tomake it easier for users to listen to speeded-up voices, it is desirableto start playback from a normal speed and increase the speed step bystep to 2×, 3×, and 4× and then reduce the speed to 3×, 2×, and finallyto 1× at the end of the playback. Thus, it is desirable that the speedof voice data stored on a CD-ROM be varied in the above-describedmanner.

When a voice is played back, it is preferable to play back a naturalsound (such as a wind sound, a wave sound, a sound of a stream, or abird's singing) at a normal speed such that the natural sound issuperimposed on the voice, because the superimposed natural sound causesthe speeded-up voice to be input into a subconscious layer of a humanbrain thereby enhancing memorization. Another advantage of superimposinga natural sound is that the human brain of a user is stimulated and theuser becomes more active in practicing. Conversely, in a case in whichno superimposed natural sound is played back, the speeded-up voicecauses the user to become nervous. A natural sound superimposed on aspeeded-up voice allows a user to easily accept the voice without havingto make an effort to consciously listen to the voice and thus withouthaving a feeling that the voice is played back at an increased speed.

Instead of using audio data for quick playback stored on a CD-ROM, ifspeeded-up voice data is produced by using a program installed on acomputer of an end user, the speed-up factor can be set to not only aninteger but also any desired value in steps of, for example, 0.5 or 0.1.How quickly users become accustomed to listening to speeded-up voicesdepends on users. Parameters such as the practice time and the speed-uppattern can be adjusted by users depending on the degree ofaccustomness.

The current multimedia technology allows sounds to be played back insynchronization with associated text information and/or imageinformation. If speeded-up voice data is played back in synchronizationwith associated with text information and/or image information,multiplier effects of the combination of quick listening and quickreading can be obtained.

FIG. 10 is a flow chart showing a process of displaying a moving imageof, for example, a bird. As for the moving image used herein, it isdesirable to employ a moving image in a scene of a natural environment.Examples of desirable moving images include a singing bird, a sea, and asmall stream. A specific desirable example is a moving image of anatural scene taken through a window of a car running in a naturalenvironment. In this case, it is more desirable that the running speedof the car is increased in synchronization with increasing speed ofvoice data.

FIG. 11 is a flow chart of a process of setting a mode. Table 1 showsspecific examples of files of natural sounds and moving images. Table 2shows an example of a folder structure in which files of natural soundsand moving images are stored. TABLE 1 Example of files file of naturalsound file of moving image bird bird.wav bird.mpeg river river.wavriver.mpge sea sea.wav sea.mpeg heart heart.wav heart.mpeg

TABLE 2 Example of folder structure Folder: Glabo Subfolder for storingnatural sounds: Backsound Subfolder for storing moving images: Movie

In the conventional technique, when speeded-up voice data is playedback, an alpha wave is produced by superimposing only a natural sound.The alpha wave can also be produced by visual information superimposedon speeded-up voice data. Thus, if a natural moving image with 1/ffluctuations is displayed on a computer screen in addition to a naturalsound, visual and audio information allows the alpha wave to be producedmore effectively. Furthermore, displaying a moving image on the computerscreen allows a user to focus his/her eyes on the image and thus toincrease his/her concentration on learning.

FIG. 12 is a flow chart showing an equalization process.

FIG. 13 is a flow chart of a process of setting equalization modes. Inthis example, a user is allowed to select one of three modes: normalmode, unborn-baby sound mode, and language learning mode. In theunborn-baby sound mode, frequency components of voice data are convertedinto a range which would be hearable by an unborn baby in a mother'swomb. In the language learning mode, equalization is performed so as toemphasize frequency components in a range within which frequencycomponents of a foreign language fall. Table 3 shows an example of dataconverted in the unborn baby sound mode so as to include only frequencycomponents higher than 8 kHz. Table 4 shows an example of data convertedin the language learning mode such that frequency components higher than2 kHz are emphasized. TABLE 3 Example of data (having only frequencycomponents higher than 8 kHz for use in the unborn baby sound mode) HzLevel F1 = 100 L1 = 0.0 F2 = 400 L2 = 0.0 F3 = 1200 L3 = 0.0 F4 = 2000L4 = 0.0 F5 = 8000 L5 = 3.0 F6 = 16000 L6 = 3.0

TABLE 4 Example of data (whose frequency components higher than 2 kHzare emphasized, for use in the language learning mode) Hz Level F1 = 100L1 = 1.0 F2 = 400 L2 = 1.0 F3 = 1200 L3 = 1.0 F4 = 2000 L4 = 1.5 F5 =8000 L5 = 2.0 F6 = 16000 L6 = 3.0

FIG. 14 shows passbands of various languages. As can be seen from FIG.14, the frequency band used in the Japanese language are significantlydifferent from those of foreign languages such as UK English or AmericanEnglish. It has recently turned out that the difference in frequencyband between the Japanese language and foreign languages makes itdifficult for Japanese people to learn foreign languages. In playingback voice data at an increased speed, if equalization is performed soas to emphasize frequency components in ranges used in foreignlanguages, the above-described difficulty in learning foreign languagescan be removed. This mode in which voice data equalized in such a manneris played back is herein referred to as a language learning mode.

In the unborn-baby sound mode, equalization is performed such thatresultant equalized voice data includes only frequency components higherthan 8 kHz. This mode simulates a state in which an unborn baby in amother's womb hear a sound/voice. According to Dr. Tomatis, a Frenchaudiologist, the unborn-baby sound mode makes it possible to reviveears.

FIG. 15 shows the relationship between parts of a human body andfrequencies of sounds. The relationship between various parts of a humanbody and frequencies of sound have been investigated. There is anattempt to make a medical treatment (known as a Mozart treatment) on thebasis of knowledge on the relationship between human parts and soundfrequencies. According to the investigation., human brains are coupledwith a sound in a 8 kHz frequency band, and thus it can be understoodthat sounds in the unborn baby sound mode apply sound energy to humanbrains.

A natural moving image and/or a natural sound may be downloaded via theInternet, and the downloaded natural moving image and/or natural sound(such as a bird's singing, a wave-sound, a sound of a small stream) maybe superimposed on voice data played back at an increased speed.

Of various natural sounds, a sound of beating of a heart (of a mother)is particularly useful to treat depression.

Referring to hard copies of screens shown in FIGS. 16 to 29, thecomputer program for realizing the method of assisting a user tomemorize or learn a language according to the present invention isdescribed below. In the present invention, the program is referred to asa flip card program.

FIG. 16 shows an initial screen which is displayed when the program isstarted. A drill selection button is disposed in the upper left cornerof the screen. If no drill is selected yet, it is necessary to click thedrill selection button to select a drill. In the example shown in FIG.16, a drill entitled “vehicles” selected in previous learning isinherited. Thus, herein, it is assumed that the drill entitled“vehicles” used in the previous memorization practice or languagelearning is employed without selecting another new drill. A “setting”button at the upper right is used to set display parameters such as acycle time (card turning rate), an image display period, the timing ofdisplaying text data, and a text display period (the timing of deleingtext data). If the setting button is clicked, a screen for setting thedisplay parameters appears. Various sets of specified conditions can beregistered. In the present example, nine sets of conditions from“condition 1” to “condition 9” have been already registered (they can becustomized), and “condition 5” is currently selected because “condition5” was used in previous execution of the program and the settings areretained. In a case in which the same conditions as those used theprevious execution are employed, it is not necessary to click the“setting” button before starting the program. An “overlap” slider isused to specify an overlap ratio between a left-hand display area and aright-hand display area on the screen (between an image displayed on theleft-hand side of the screen and text information displayed on theright-hand side of the screen). The slider may be slid by dragging itwith a mouse. If a pointer is placed on the slider by using the mouse, anumeral indicating a current overlap ratio is displayed in a pop-upfashion. In the example shown in FIG. 16, the overlap ratio is set to30%. When the slider is set to a leftmost position, the overlap ratio isset to 0%, while the overlap ratio is set to 100% when the slider is setto a rightmost position. When the overlap ratio is set to 0%, an imageis displayed in a left-hand display area with a width one-half the totalwidth of the screen. In the case in which the overlap ratio is set to100%, the image display area extends over the entire width of the screenand an image is displayed at the center of the display area. Buttons of“normal”, “2×”, “3×”, “4×”, “5×”, “6×”, “7×”, and “8×” are used tospecify the speed-up factor of the playback speed of a flip card voice.A “close” button is used to end the flip card system.

A “BGM” button is disposed below. the “overlap” slider. In the exampleshown in FIG. 16, no background music (BGM) is selected. A desirednatural sound such as a sound of a small stream, a wave sound, or abird's singing or desired music such as baroque music or Mozart's musiccan be selected. Background music is effective to provide a goodenvironment in which an alpha wave is easily generated in a user'sbrain. If learning of a language is performed in a relaxed state inwhich an alpha wave is generated, learning is performed effectively. Inan area right to the “BGM” button, there are radio buttons labeled“normal”, “2×”, “3×”, “4×”, “5×”, “6×”, “7×”, and “8×”. There are alsoradio buttons labeled “Japanese”, “Korean”, “Chinese”, “Russian”,“English”, “French”, “German”, “Spanish”, “Portuguese”, and “Greek”, forselecting a language. It is possible not only to display textinformation linked to an image but also to play back a voice reading thetext information. The voice reading text information is not in a singlelanguage such as Japanese but in a plurality of languages includingother languages and voice data in those languages are stored inconnection with a corresponding image. In a case in which a voice with anormal speed is played back, the length of a period in which the voiceis played back limits the card turning rate (periodic intervals at whichimages are displayed one after another). Another advantage,experimentally demonstrated, of playing back a voice at a speedincreased by a factor of 3 or 4 is that the increased speed stimulates ahuman brain thereby causing the brain to become more active. To make itpossible to turn cards at a higher speed (display images at shorterintervals) and to visually and audibly stimulate a human brain, thepresent program plays back voice data at an increased speed. If voicedata are previously produced for various speed-up factors and resultantdata are stored, it is necessary to store a large amount of data. Toprevent the above problem, only voice data with a normal speed isstored, and speeded-up voice data is produced when it is necessary (asdisclosed in Japanese Unexamined Patent Application Publication No.2001-282298, entitled “Method of producing voice data, method of playingback voice data, and storage medium”). Speeded-up voice data can beproduced without causing a problem in execution of the flip card. Forexample, a routine of producing speeded-up voice data may bepre-executed. More specifically, if “4×” is selected as the speed-upfactor and “Japanese” is selected as the language in the initial screenshown in FIG. 16, a set of speeded-up voice data is immediately producedfor a necessary set of voice data. When producing of the quick set ofspeeded-up voice data is completed, execution of the flip card isstarted.

The initial screen shown in FIG. 16 has a “start” button and a “close”button. If the “start” button is clicked, execution of the flip cardprogram is started (when it is necessary to produce a set of speeded-upvoice data for a set of voice data, the execution of the flip cardprogram is started after completion of producing the set of speeded-upvoice data). The “close” button is used to terminate the flip cardprogram.

FIG. 17 is a hard copy of a setting screen which is displayed when thesetting button is clicked. On this setting screen, sliders for settingfour parameters associated with “display cycle time”, “image displayperiod”, “text display time”, and the “text delete time” are provided.To set parameters using the slider, an operator of a computer grasps oneof sliders with a mouser and drags it to a desired position (theoperator of the computer may be a user of the computer who memorizes orlearns a language using the flip flop program running on the computer orthe operator may be a teacher of a school or a parent). The “displaycycle time” is a repetition period of displaying images. In the presentexample, the display period is set to be equal to 0.5 sec. If thedisplay period is decreased, the card turning rate increases. Thedisplay period may be set to be as small as, for example, 0.3 sec. In acase in which the display cycle time is set to 0.5 sec, the lengths ofthe horizontally extending sliders for setting the “image displayperiod”, the “text display time”, and the “text delete time” are all setso as to correspond to 0.5 sec. The slider for setting the “imagedisplay period” is used to specify how long to display an image afterstarting displaying the image at the beginning of the display period. Inother words, the position of this slider indicates the timing ofdeleting the image. The slider for setting the “text display time” isused to specify the timing of starting displaying language textinformation on the right-side of the screen. Preferably, displaying oflanguage text information is started after a very short pause after thecompletion of displaying the image. The slider for setting the “textdelete time” is used to specify the timing of deleting the textinformation. In other words, by specifying the text delete time usingthis slider, the length of a period during which the text information isdisplayed is-determined. In a bottom area of the screen shown in FIG.17, there is a field for selecting a setting number. If an invertedtriangular button on the right end of this field is clicked, a list ofsetting numbers (such as setting 1, setting 3, setting 4, and so on) isdisplayed. If one of a setting number is selected from the list, displayparameters defined in the selected setting number are displayed. If the“setting” button is clicked, the parameters specified in this screen aresaved in connection with the selected setting number, and the settingscreen is closed. If the “cancel” button is clicked, the setting screenis closed without saving the display parameters.

FIG. 18 is a hard copy of a screen on which an image of a “double-deckerbus” is displayed on the left-hand side. In this specific example, theleft-right image overlapping ratio is set to 30%.

FIG. 19 is a hard copy of a screen on which text data of “double-deckerbus” is displayed on the right-hand side. In this example, theleft-right image overlapping ratio is set to 30%, and Japanese isspecified as the language.

FIG. 20 is a hard copy of a screen on which an image of a “horse-drawncarriage” is displayed on the left-hand side. In this example, theleft-right image overlapping ratio is set to 0%.

FIG. 21 is a hard copy of a screen on which text data of “horse-drawncarriage” is displayed on the right-hand side, wherein the left-rightimage overlapping ratio is set to 0%, and Japanese is specified as thelanguage.

FIG. 22 is a hard copy of a screen on which an image of a “jeep” isdisplayed on the left-hand side, wherein the left-rightimage-overlapping ratio is set to 100%.

FIG. 23 is a hard copy of a screen on which text data of “jeep” isdisplayed on the right-hand side, wherein the left-right imageoverlapping ratio is set to 100%, and Japanese is specified as thelanguage;

FIG. 24 is a hard copy of a screen on which an image of a “scooter” isdisplayed on the left-hand side, wherein the left-right imageoverlapping ratio is set to 71%.

FIG. 25 is a hard copy of a screen on which text data of “scooter” isdisplayed on the right-hand side, wherein the left-right imageoverlapping ratio is set to 71%, and Japanese is specified as thelanguage.

FIG. 26 is a hard copy of a screen on which an image of a “van” isdisplayed on the left-hand side, wherein the left-right imageoverlapping ratio is set to 16%.

FIG. 27 is a hard copy of a screen on which text data of “van” isdisplayed on the right-hand side, wherein the left-right imageoverlapping ratio is set to 16%, and Japanese is specified as thelanguage.

FIG. 28 is a hard copy of a screen on which an image of a “cable car” isdisplayed on the left-hand side, wherein the left-right imageoverlapping ratio is set to 82%.

FIG. 29 is a hard copy of a screen on which text data of “cable car” isdisplayed on the right-hand side, wherein the left-right imageoverlapping ratio is set to 82%, and Japanese is specified as thelanguage.

FIG. 30 is a flow chart showing an operation of the “flip card” programaccording to the present invention. After initial settings such asselection of a drill, selection of a card turning rate, selection ofbackground natural sound (music), selection of a speed-up factor of theplayback speed, selection of a language, and selection of an overlapratio (between images and text information) are completed, the programis started. In execution for a second or later time, if it is allowed touse the same settings as those employed in previous execution, theprogram may be started without performing the initial setting process.If the “start” buttons is clicked, the program is started. After theprogram is started, drill data and display parameters are acquired.Thereafter, if it is specified that background music (or a naturalsound) should be played back, playing back of background music isstarted. A card is then turned. Image data and text and voice datalinked to the image data are read, and displaying of the image data andtext data and playing back of the voice data are performed in accordancewith the display parameters. It is determined whether there is moreimage data. If there is no more image data, the program is ended.However, if the image data is not last one, it is determined whethersetting is changed. If setting is not changed, the card is turned todisplay a next card. On the other hand, in the case in which setting ischanged, after new parameters are acquired, the card is turned,displaying and playing back are performed in accordance with the newparameters.

FIG. 31 is a flow chart showing a process performed by a user to setvarious conditions. In this setting process, the user selects a drill, acard turning rate (display cycle time), a background natural sound(background music), a speed-up factor of the playback speed, and anoverlapping ratio. The setting may be performed by dragging sliders witha mouse or by making selections from lists.

FIG. 32 is a time chart associated with an operation performed when 2×voice data has a length shorter than a specified card turning rate. Theplaying back of the natural sound (background music) is continuouslyperformed regardless of the card turning rate (image display period).The card turning rate (repetition period of displaying images) may beset by the operator of the computer to a desired value such as 0.3 sec,0.5 sec, or 0.8 sec. In FIG. 32, the card turning rate is designated asTturn. Basically, cards are turned and images are displayed one afteranother every Tturn. In a case in which displaying of an mage,displaying of text information, and playing back of a voice arecompleted by the end of each display period Tturn, the operation isperformed as specified by the setting conditions, as shown in FIG. 32.However, although displaying of an image and displaying of textinformation can be completed by the end of each display period Tturnregardless of the specified conditions, there is a possibility thatplayback of a voice cannot be completed by the end of the display periodTtutn. For example, if the specified value of Tturn is too small, or ifa speed-up factor of the playback speed is small and thus it takes along time to play back a voice, playback of the voice cannot becompleted by the end of Tturn. In such a case, exception handling isperformed as shown in FIG. 33.

FIG. 33 is a time chart associated with an operation performed when 2×voice data has a length longer than a specified card turning interval.In this case, the card turning interval is automatically increased bythe program. More specifically, turning to a next card (displaying anext image) is performed after playback of a voice is completed.

FIG. 34 is a conceptual diagram showing a method of assisting a user tomemorize or learn-a language according to the present invention. Asdescribed earlier, the left-hand part of the brain is responsible forprocessing of languages, and the right-hand part of the brain isresponsible for processing of images. Furthermore, the right-hand partof the brain is associated with left-hand parts of a human body, whilethe left-hand part of the brain is associated with right-hand parts ofthe human body. The inventor of the present invention believes that thebrain has a particular part (bottom part) responsible for consciousoperations and a part (upper part) responsible for subconsciousoperations. In Japanese Unexamined Patent Application Publication No.2001-282298 cited earlier, the inventor of the present invention hasdisclosed a technique of removing a barrier between consciousness andsubconsciousness by playing back a quick voice while superimposing anatural sound thereon. By using the flip card in conjunction withplayback of a quick voice while superimposing a natural sound, it ispossible to train the brain to operate in such a manner that theleft-hand part and the right-hand part of the brain effectivelycooperate with each other.

FIG. 35 is a diagram conceptually showing image data, text data, andvoice data, which are linked to each other and used in the method ofassisting a user to memorize or learn a language according to thepresent invention. Each image data such as an image of a mountain or animage of a flower has a plurality of category codes which allow imagedata to be easily retrieved. The category codes include a serial number,a main category (attribute), a middle category (attribute), and a minorcategory (attribute). Each image is linked to text information in aplurality of languages such as Chinese, Korean, Japanese, English,Russian, and French. According to the category codes, it is possible toextract similar words and/or associated words such as antonyms, orsynonyms. The category codes can also be used to extract words accordingto a particular property such as a grade or a level. For example,first-type categories may include 160 categories such as natural worlds,materials, children, couples, lives, families, marriage, occupations,mass communications, businesses, politics, and science. Each first-typecategory may include second-type categories (sub categories). Forexample, the first-type category of natural worlds may include 16second-level categories such as mountains, seas, rivers, and flowers,and the first-type category of materials may include 16 second-levelcategories such as chemical materials, stone, wood, and plastics.Third-type categories may be defined from a grammatical point of view.For example, the third-type categories may include adjective, adverbs,illustrative sentences, and short sentences. Each piece of language textinformation may include not only a word but also a sentence such as ashort sentence or an illustrative sentence. Fourth-type categories maybe defined according to grades in school. For example, fourth-typecategories may include preschool grade, elementary school grade, juniorhigh school grade, senior high school grade, university grade, andgraduate grade. Fifth-type categories may be defined globally for alllanguages according to degrees of difficulty. For example, thefifth-type categories may include 12 categories such as level 1, level2, level 3, level 4, level 5, level 6, . . . , level 12.

FIG. 36 is a diagram showing a screen for selecting a CD storing music(or a natural sound) to be played back in execution of the method ofassisting a user to memorize or learn a language according to thepresent invention. In this specific example shown in FIG. 36, a message“NO MUSIC CD” is displayed to indicate that a selectable music CD is notfound or a content of a CD is not of music although the CD is found. Inthis situation, buttons “repeat all”, “repeat single piece”, and“shuffle” are all disabled.

FIG. 37 is a diagram showing a screen for selecting a CD storing music(or a natural sound) to be played back in execution of the method ofassisting a user to memorize or learn a language according to thepresent invention. In this specific example shown in FIG. 37, a music CDis found and the number of pieces stored on the music CD is displayed(20 pieces are stored on the music CD, in the example shown). In thisstate, if the “play” button is clicked, all pieces of music stored onthe CD are played back successively.

FIG. 38 is a diagram showing a screen for selecting a CD storing music(or a natural sound) to be played back in execution of the method ofassisting a user to memorize or learn a language according to thepresent invention, wherein in this specific example, an invertedtriangular button on the right-hand end of a field in which a message“20 pieces of music is stored on a music CD” is indicated is clicked toopen a pull-down list of selectable 20 pieces of music. The user canspecify a desired piece of music by selecting a corresponding track fromthe list.

FIG. 39 is a diagram showing a screen for selecting a CD storing music(or a natural sound) to be played back in execution of the method ofassisting a user to memorize or learn a language according to thepresent invention, wherein in this specific example, music with a lengthof 2 min 47 sec stored on track 3 is selected, and a “repeat singlepiece” button becomes clickable. After specifying a track, if the“repeat single piece” button is clicked and further the “play” button isclicked, the selected piece of music is played back.

FIG. 40 is a diagram showing a screen for selecting a CD storing music(or a natural sound) to be played back in execution of the method ofassisting a user to memorize or learn a language according to thepresent invention, wherein in this specific example, all 20 pieces ofmusic are selected and a “shuffle” button is clicked to specify that theselected 20 pieces of music should be played back at random. That is, ifthe “play” button is clicked after the “shuffle” button is clicked, allpieces of music are played back in a shuffle mode (at random).

FIG. 41 is a diagram showing files used in the program of the method ofassisting a user to memorize or learn a language. When installation ofthe flip card system according to the present invention and image datais completed, a folder named “ClipArts” for storing image data and voicedata is created. The folder “ClipArts” has sub-folders “animals” and“vehicles” each of which has a sub-folder “voice” for storing voicedata, and the sub-folder “voice” has a sub-folder “2× voice” for storing2× voice data.

FIG. 42 is a diagram showing a screen which is displayed when theprogram of the method of assisting a user to memorize or learn alanguage is started. This is an example different from that shown inFIG. 16. In this example shown in FIG. 42, the image-text overlappingratio is set to 100%. In this screen, a “drill selection” button is usedto select a drill, and the title of a selected drill is displayed in a“drill title” field. A “setting selection” box is used to specifyparameters such as a display cycle time. A “setting” button is used toopen a screen for setting display parameters. An “overlap” slider isused to specify the overlap ratio (in %) between an image and text. A“start” button is used to start displaying the flip card program. A“BGM” button is used to select CD music to be played back as backgroundmusic. A “BMG number” indicates a track number of background musiccurrently selected.

Setting associated with a drill are described below with reference toFIGS. 43 to 45. FIG. 43 is a hard copy of a screen for performingsettings associated with a drill. To add a new image, an “add” button isfirst clicked. In response, an “open” screen appears, which allows auser to add a new image to the list of images, at a position immediatelybelow a highlighted position. In this specific example, “truck” is addedbelow “scooter”. Image may be stored in image files in a proper formatsuch as gif, jpg, or bmp.

FIG. 44 is a hard copy of a window in which a list of drills isdisplayed. In this window, a user can select a drill list file to beopened. Herein, each drill list file has a file name in the form of“***.lst”. The file name is directly employed as a drill title.

FIG. 45 is a diagram showing a drill setting screen. In this screen, auser can call image data and can input text data to be linked to theimage data. It is also possible to link desired music data to the imagedata. In this screen an “open” button is used to open an existing drilllist. A “create” button is used to create a new drill list. In a “drilltitle” filed, a drill title is displayed. A “save as” button is used tospecify a name of a file into which to store a drill list. A “return”button is used to return to the main screen. An “add” button is used toadd an image to an image list. A “delete” button is used to delete aspecified image from a list. An “up” button is used to move the locationof a specified image upwardly by one position. A “down” button is usedto move the location of a specified image downwardly by one position.When the flip card system is executed for the first time, no drill nameis specified. In this case, after completion of installing image files,the “open” button is clicked to open a desired file such as“vehicles.lst” in the directory “ClipArt”, and setting associated withthe drill is performed.

Display parameters are described below with reference to FIG. 47. FIG.47 is a hard copy of a screen for setting conditions associated with adisplay cycle time, an image display period, a text display time, and atext delete timing. In this screen, a “display cycle time” slider isused to specify periodic intervals at which flip cards are displayed oneafter another. A shortest allowable display cycle time is 0.3 sec. An“image display period” slider is used to specify a period during whichto display an image, wherein the image display period is represented in% relative to the display cycle time specified by the “display cycletime” slider. A “text display time” slider is used to specify a time atwhich displaying of text information is started, wherein the time isrepresented in % relative to the display cycle time. Playback of acorresponding voice is also started at the same time as the text displaytime. A “text delete time” slider is used to specify a time at which thetext information is deleted. However, if a corresponding voice is playedback further after the specified text delete time, the text informationis also further displayed. That is, the length of the period ofdisplaying text information varies depending on the length of the periodof playing back corresponding voice data. A “setting number” selectionbox is used to select a setting number assigned to a set of parameters.A “setting” button is used to apply the specified parameters to aselected setting number. A “cancel” button is used to return to the mainscreen-without changing any parameter.

FIG. 48 is a diagram showing an initial screen for installing onto acomputer the program of the method of assisting memorization accordingto the present invention. After a CD-ROM on which the flip card programis stored is opened, if setup.exe is executed by double-clicking, theinitial screen shown in FIG. 48 appears. If an “OK” button is clicked,the installation is started.

FIG. 49 is a diagram showing a screen for specifying a directory intowhich to install the program. When it is desired to change the directoryinto which to install the flip card program, a “change directory” buttonis clicked and a directory is specified. After specifying the directory,if a “setup” button is clicked, the installation is started.

FIG. 50 is a diagram showing a screen for specifying. a program groupname into which to register the flip card system.

FIG. 51 is a diagram showing a window in which a message indicating thecompletion of the installation is displayed. At this point of time, theinstallation of the program has been completed. Thereafter, installingof image data is performed.

FIG. 46 is a diagram showing a screen for installing image data or voicedata to be used in the flip card system. This screen can be opened bysuccessively selecting “start”, “program”, “flip card”, and “installimage data”, from menus.

FIG. 52 is a diagram showing a manner in which the program is started.As shown in FIG. 52, the program can be started by selecting “flip card”from the “start” menu.

FIG. 53 is a diagram showing a manner in which the program isuninstalled. When the flip card system is deleted or upgraded, theexisting flip card system is uninstalled. After a control panel isopened, if “add/delete applications” is selected in the control panel, alist of applications is displayed. If “flip card” is selected from thelist, and an “add/delete” button is clicked, the flip card program isuninstalled.

FIG. 54 is a diagram showing a screen in which condition 1 is set. Inthis specific example, the display cycle time is set to 0.4 sec andother conditions are set as follows. At the beginning of each displaycycle time, an image is displayed for the image display period equal to20% of the display cycle time. After deleting the image at the end ofthe image display period, a short pause is put. After the short pause,text data is displayed for the text display period equal to 10% of thedisplay cycle time. At the end of the text display period, the text datais deleted.

FIG. 55 is a diagram showing a screen in which condition 2 is set. Inthis specific example, the display cycle time is set to 0.5 sec andother conditions are set as follows. At the beginning of the displaycycle time, an image is displayed for the image display period equal to20% of the display cycle time. After deleting the image at the end ofthe image display period, a short pause is put. After the short pause,text data is displayed for a period equal to 40% of the display cycletime.

FIG. 56 is a diagram showing a screen in which condition 3 is set. Inthis specific example, the display period is set to 0.3 sec and otherconditions are set as follows. At the beginning of the display cycletime, an image is displayed for a period equal to 50% of the displaycycle time. Text data is displayed for a period equal to 40% of thedisplay cycle time, partially overlapping with the image display period.

FIG. 57 is a diagram showing a screen in which condition 4 is set. Inthis specific example, the display cycle time is set to 1.8 sec, andother conditions are set as follows. At the beginning of the displaycycle time, an image is displayed for a period equal to 5% of thedisplay cycle time. After the image display period, a short pause isput. After the short pause, text data is displayed for a period equal to50% of the display cycle time.

FIG. 58 is a diagram showing a screen in which condition 5 is set. Inthis specific example, the display cycle time is set to 2.9 sec, andother conditions are set as follows. An image is displayed for a periodequal to 60% of the display cycle time, and text data is displayed for aperiod equal to 20% of the display cycle time following the imagedisplay period.

FIG. 59 is a diagram showing a screen in which condition 6 is set. Thedisplay cycle time is set to 2.6 sec, and an image is displayed for aperiod equal to 50% of the display cycle time. Text data is displayedfor a very short period.

FIG. 60 is a diagram showing a screen in which condition 7 is set. Thedisplay cycle time is set to 0.5 sec. An image is displayed for a periodequal to 50% of the display cycle time, and an image is displayed for aperiod equal to 20% of the display cycle time.

FIG. 61 is a diagram showing a screen in which condition 8 is set. Thedisplay cycle time is set to 0.5 sec. An image is displayed for a periodequal to 10% of the display cycle time, and text data is displayed for avery short period.

FIG. 62 is a diagram showing a screen in which condition 9 is set; Thedisplay cycle time is set to 1 sec. An image is displayed for a periodequal to 30% of the display cycle time, and an image is displayed for aperiod equal to 20% of the display cycle time.

Although sets of conditions are defined as default, they can be modifiedby an operator. For example, conditions may be modified depending on thedegree of acustomness of a user of the system or depending on the degreeof progress in learning.

Although in the embodiments described above, it is assumed that amachine using basic software provided by Microsoft Corp. in the USA isused as the computer, another type of machine such as a personal digitalassistance (PDA) device or a video game machine may be employed. In somecases in which a portable video game machine is employed, a music CDcannot be used to playback background music. In such a case, music datarepresented in mpg3 or a similar format may be used to playbackbackground music. In a case in which a machine using Microsoft Windowsis employed as the computer, a user of the computer may add images andstore text data and/or voice data in relation with respective imagesthereby growing the image database in a customized manner. However, in acase in which a game machine is used as the computer, customization islimited, depending on the machine.

Although in the embodiments described above, images of animals orvehicles are displayed by example, the invention may be applied tolearning of not only words representing concrete objects but also wordsrepresenting abstract concepts. In learning languages using the systemaccording to the present invention, when a word can correspond in manyways to images, a representative image is selected and the word islinked to the selected image. In the conventional technique of learninglanguages, a dictionary indicating correspondence between words in onelanguage and words in another language is used. In contrast, the presentmethod of assisting users to memorize or learn languages makes itpossible to learn a plurality of languages via images rather thancorrespondence. Thus, in this method, learning is performed efficientlyusing not only the left-hand part of the human brain of user but alsoother parts.

As described above, the present invention has great advantages. That is,the present invention makes it possible to achieve a multimedia-basedquick playback system. Furthermore, the invention also makes it possibleto achieve a quick playback system in which combinations between voicesand corresponding images are effectively used. The quick playback systemaccording to the present invention makes it possible to memorize orlearn languages using not only either one of the left-hand part andright-hand part of the human brain but using both parts and also usingboth conscious and subconscious activities of the brain.

1.-31. (canceled)
 32. A computer-assisted method of assisting a user to memorize, comprising the steps of: displaying an image indicating a content to be memorized, on a left-hand side of a computer screen for a predetermined period of time; displaying text data indicating language information related to the image displayed in the image display step, on a right-hand side of the computer screen for a predetermined period of time; and playing back a voice pronouncing the text data displayed in the text data displaying step, wherein the above three steps are performed repeatedly to successively display a plurality of images and text data and play back voices thereby assisting the user of the computer to memorize.
 33. A method of assisting a user to memorize, according to claim 32, further comprising a time parameter adjustment step of adjusting, by the user of the computer, a cycle time of repeatedly displaying images in the image display step, the length of an image display period during which each image is displayed in the image display step, the timing of starting displaying text data in the text display step, and the length of a text display period during which text data is displayed in the text data displaying step, wherein once the adjustment in the time adjustment step is performed by the user of the computer, displaying of the image in the image display step and displaying of the text data in the text data display step are performed in accordance with the conditions set in the time adjustment step.
 34. A method of assisting a user to memorize, according to claim 32, further comprising a time parameter adjustment step of adjusting, by the user of the computer, a cycle time of repeatedly displaying images in the image display step, the length of an image display period during which each image is displayed in the image display step, the timing of starting displaying text data in the text display step, the length of a text display period during which text data is displayed in the text data displaying step, and the timing of starting playing back the voice in the voice playback step, wherein once the adjustment in the time adjustment step is performed by the user of the computer, displaying of the image in the image display step, displaying of the text data in the text data display step, and playback of the voice in the voice playback step are performed in accordance with the conditions set in the time adjustment step.
 35. A method of assisting a user to memorize, according to claim 32, further comprising a voice playback speed adjustment step of adjusting, by the user of the computer, the speed of playing back the voice in the voice playback step, wherein once the adjustment in the voice playback speed adjustment step is performed by the user of the computer, playback of the voice in the voice playback step is performed in accordance with the condition set in the adjustment step.
 36. A method of assisting a user to memorize, according to claim 32, further comprising an area overlap ratio setting step of setting, by the user of the computer, an area overlap ratio indicating the degree of overlapping between an area in which images are displayed in the image display step and an area in which text data is displayed in the text display step, wherein once the setting in the area overlap ratio setting step is performed by the user of the computer, displaying of the image in the image display step and displaying of the text data in the text data display step are performed in accordance with the condition set in the setting step.
 37. A method of assisting a user to memorize, according to claim 35, further comprising the step of playing back a natural sound or music such that the natural sound or music is superimposed on the voice played back at the adjusted speed in the voice playback step.
 38. A method of assisting a user to memorize, according to claim 32, wherein the plurality of image data are stored in the form of a database; the method further comprises the step of extracting image data categorized by synonym, antonym, grade, type, or the like; and the extracted image data are displayed in the steps performed repeatedly. 39.-48. (canceled)
 49. A computer program for causing a computer to execute a method of assisting a user to memorize according to one of claims 32, 33, 34, 35, 36, 37, or
 38. 50. A method of assisting a user to memorize according to one of claims 32, 33, 34, 35, 36, 37, or 38, wherein the computer is included in a personal digital assistance (PDA) device, a game machine, or a similar device. 51.-52. (canceled)
 53. A method of assisting a user to memorize according to one of claims 32, 33, 34, 35, 36, 37, or 38, a further comprising an image data modification step performed by the user to add, change, and/or delete image data thereby allowing the user to modify the image data so as to adapt to progress of memorization.
 54. (canceled) 